Systems and methods for audio capture

ABSTRACT

A method for noise filtering including receiving directional data corresponding to the relative directions of at least one noise source and at least one target audio source; capturing noise data from the at least one noise source; capturing target audio data from the at least one target audio source; using the directional data to filter the noise data from the target audio data; and outputting filtered target audio.

FIELD

This invention relates to systems and methods for audio capture. Moreparticularly, but not exclusively, the invention relates to systems andmethods for noise filtering in audio capture.

BACKGROUND

Many aircraft, such as unmanned aerial vehicles (UAVs), helicopters,vertical lift systems and fixed-wing aircraft, disadvantageously producenoise. In UAVs, noise may be produced by an engine (due to, for example,the exhaust or combustion), a motor assembly (due to, for example,vibration), interaction of airflow with the UAV and/or the UAV'spropellers. In audio capturing UAVs, the noise produced by the UAVitself may be significantly greater than a target audio signal and thenoise may prevent or impede the UAV's audio capture or processing.

Noise produced by UAVs is a particular problem for UAVs used for videoand audio capture for filming. Such filming may be for livebroadcasting, recording events (for example, concerts), or it may be forentertainment and documentary purposes (such as filming for televisionor movies).

Current UAV audio capture for filming requires expensive andtime-consuming post-processing to remove noise produced by the UAVs.Typically, audio is captured during UAV filming by having a microphoneon the ground and/or by having the target of interest wearing a separatemicrophone. This has the disadvantage of UAV noise being picked up bythe microphones on the ground and/or the target of interest. Thisrequires expensive and time-consuming post-processing. Further, itrequires ground or body microphones to be set up, limiting versatility.

Other areas in which UAV noise is a concern include UAV audio capture indefence and security, law enforcement, industrial, and remotecommunication applications. UAVs are well suited for such applicationsbecause they are quickly deployable, they may be deployed remotely andthey may cover significant distances. Audio capture may be used toidentify targets (for example, through spectral analysis), for acousticsource localization or to measure noise levels. In defence and securityapplications, audio may be captured for gunfire detection. In industrialapplications, audio captured by UAVs may be used to detect mechanicalfaults and evaluate noise compliance. UAVs may be used to allow remotecommunication. For example in search and rescue, UAVs may be used tocapture audio from a survivor in a remote location (and transmit this tosearch and rescue personnel). In another example, in logistics (such aspackage delivery) audio from a recipient may be captured (andtransmitted to the delivery company). Hobby/recreational/selfie UAVusers may also wish to record audio.

The present invention may provide improved audio capture or at leastprovide the public or industry with a useful choice.

SUMMARY

According to one example embodiment there is provided a method for noisefiltering including: receiving directional data corresponding to therelative directions of at least one noise source and at least one targetaudio source; capturing noise data from the at least one noise source;capturing target audio data from the at least one target audio source;using the directional data to filter the noise data from the targetaudio data; and outputting filtered target audio.

The directional data may be used to determine at least one beamformingconfiguration wherein at least one beam associated with the beamformingconfiguration captures at least one of the noise source or the targetaudio source. The method may include the step of applying thebeamforming configuration to sound data captured by the sound capturingdevice.

The noise data and target audio data may be captured using a soundcapturing device, and the directional data may be used to estimate thepower of the noise data and/or target audio data at the sound capturingdevice.

The sound capturing device may include a target audio capturing deviceand a noise capturing device, and the directional data may be used toestimate the power of the noise data and/or target audio data at thetarget audio capturing device.

According to another example embodiment, there is provided a system foran unmanned aerial vehicle (UAV) including: a sound capturing deviceconfigured to capture noise data from at least one noise source and tocapture target audio from at least one target audio source; and aprocessing unit configured to: receive directional data corresponding tothe relative directions of the at least one noise source and the atleast one target audio source; receive the noise data and the targetaudio data; use the directional data to filter the noise data from thetarget audio data; and output filtered target audio.

The directional data may be used to estimate the power of the noise dataand/or target audio data at the sound capturing device.

The system may include a directional sensor configured to sense thedirection of the noise source relative to the UAV and the direction ofthe target audio source relative to the UAV, and the processing unit maybe configured to determine a relative direction between the noise sourceand the target audio source.

The sound capturing device may include a noise capturing deviceconfigured to capture the noise data and a target audio capturing deviceconfigured to capture the target audio.

The system may further include at least one sensor and the processingunit may be configured to receive sensor data from the at least onesensor and associate the filtered target audio with the sensor data. Theat least one sensor may include a video camera configured to capturevideo data and the processing unit may be configured to associate thefiltered target audio with the video data. A direction of the targetaudio capturing device may be aligned with a direction of the videocamera. The at least one sensor may be a positional sensor, and theprocessing unit may be configured to associate the filtered target audiowith positional data.

The sound capturing device may include a MEMS microphone. The soundcapturing device may be an array of microphones.

The sound capturing device may be attached to the UAV via a gimbal.

According to a yet further example embodiment, there is provided amethod for noise filtering including: receiving directional datacorresponding to the relative directions of at least one noise sourceand at least one target audio source; steering a sound capturing deviceto capture noise data from the at least one noise source and to capturetarget audio data from the at least one target audio source; using thedirectional data to filter the noise data from the target audio data;and outputting filtered target audio.

The step of steering the sound capturing device may include applying abeamforming configuration to redirect at least one beam to capture theat least one noise source and/or the at least one target audio source.

The sound capturing device may be mounted via a gimbal and the step ofsteering the sound capturing device may include steering the gimbal toredirect the sound capturing device.

The sound capturing device may include a noise capturing device tocapture the noise data and a target audio capturing device to capturethe target audio, and wherein the target audio capturing device may bemounted via a gimbal and the step of steering the sound capturing devicemay include steering the gimbal to redirect the target audio capturingdevice towards the target audio source.

According to another example embodiment, there is provided a system fornoise filtering for an unmanned aerial vehicle (UAV) including: a soundcapturing device configured to capture noise from at least one noisesource and configured to capture target audio from at least one targetaudio source, wherein the sound capturing device is steerableindependently from the UAV; and a processing unit configured to: receivedirectional data corresponding to the relative directions of the atleast one noise source and the at least one target audio source; receivethe noise data and the target audio data; use the directional data tofilter the noise data from the target audio data; and output filteredtarget audio.

The sound capturing device may be mounted to the UAV via a gimbal andthe sound capturing device may be steered by steering the gimbal.

The sound capturing device may be configured to be steered independentlyfrom the UAV by beamforming.

The sound capturing device may include a noise capturing device tocapture the noise data and a target audio capturing device to capturethe target audio, and the target audio capturing device may be mountedto the UAV via a gimbal and the target audio capturing device may besteered by steering the gimbal.

The sound capturing device may include a MEMS microphone. The soundcapturing device may be an array of microphones.

According to another example embodiment, there is provided an unmannedaerial vehicle (UAV) payload including a system as described above.

According to a further example embodiment there is provided a method ofnoise filtering including:

determining a beam forming pattern comprising a main beam and a nullbeam;

capturing noise data using the null beam;

capturing target audio data using the main beam;

filtering the noise data from the target audio data; and

outputting filtered target audio.

The null beam may captures all of the significant noise sources.

The null beam may have a beam width of at least 1800.

The null beam may have a beam width of 360°−X°, where X is the beamwidth of the main beam.

The null beam may have a gain which varies by less than 20% across it'sbeam width.

The null beam may have a frequency range, defined by a lower frequency,where the gain which varies by less than 20% at the lower frequency.

The noise data may be filtered from the target audio data uses asimplified 2×2 gain matrix, which characterises the gain of the mainbeam in the forward direction and reverse direction, and the null beamin the forward direction and reverse direction.

A gimbalised microphone mey be configured to use the method above.

The microphone may be a MEMS array.

The microphone may alternatively be an end-fire line microphone array.

One or more sensors may detect directional data corresponding to therelative directions of the gimbalised microphone and a mounting to aunmanned aerial vehicle (UAV); and a processing unit may to use thedirectional data to filter the noise data from the target audio data.

Alternatively the noise data may be filtered from the target audio datawithout a relative direction of the gimbalised microphone and a mountingto a unmanned aerial vehicle (UAV).

An unmanned aerial vehicle (UAV) may include a gimbalised microphone asabove.

It is acknowledged that the terms “comprise”, “comprises” and“comprising” may, under varying jurisdictions, be attributed with eitheran exclusive or an inclusive meaning. For the purpose of thisspecification, and unless otherwise noted, these terms are intended tohave an inclusive meaning—i.e., they will be taken to mean an inclusionof the listed components which the use directly references, and possiblyalso of other non-specified components or elements.

Reference to any document in this specification does not constitute anadmission that it is prior art, validly combinable with other documentsor that it forms part of the common general knowledge.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings which are incorporated in and constitute partof the specification, illustrate embodiments of the invention and,together with the general description of the invention given above, andthe detailed description of embodiments given below, serve to explainthe principles of the invention, in which:

FIG. 1 is a block diagram of a UAV including a system for noisefiltering;

FIG. 2 is a block diagram of the system for noise filtering in anexample context;

FIGS. 3a & 3 b are polar patterns of a sound capturing device accordingto one embodiment;

FIGS. 4a & 4 b are polar patterns of a sound capturing device accordingto another embodiment;

FIG. 5 is a block diagram of a UAV;

FIG. 6 a flow diagram of noise filtering according to one embodiment;

FIG. 7 a flow diagram of noise filtering according to one embodiment;

FIG. 8 a flow diagram of noise filtering according to one embodiment;

FIG. 9 a schematic diagram of noise filtering according to oneembodiment;

FIG. 10 a schematic diagram of noise filtering according to oneembodiment;

FIG. 11 a schematic diagram of noise filtering according to oneembodiment;

FIG. 12 a diagram of noise filtering according to one embodiment; and

FIG. 13 a diagram of noise filtering according to one embodiment;

DETAILED DESCRIPTION

The systems, methods and devices described herein provide improvedtarget audio capture using unmanned aerial vehicles (UAVs). For thepurposes of this description, ‘target audio’ will be used to refer tothe sound that is intended to be captured. For example, target audio mayinclude the speech of a person being filmed, the ambient sound of ascene being filmed, or the sound coming from an industrial location.‘Noise’ will be used to refer to unwanted and/or background sound thatis not the target audio. While this will primarily include soundproduced by the UAV itself (for example, sound from the motor and/orpropeller assembly of the UAV), it may not be limited in this respect.For example, it may also include background sounds in a scene beingfilmed, or ambient environmental sounds (when such sounds are notintended to be captured).

Before describing the methods for noise filtering, it is helpful to givean overview of the system. FIG. 1 shows a block diagram of a UAV 200including a system 100 for noise filtering. The system 100 may belocated on or about the UAV 200. The system 100 may be configured as apayload that can be removably or permanently mounted to a UAV. Thesystem may also be incorporated partially or completely into the UAVitself.

The system may include a sound capturing device, which may comprise atarget audio capturing device 102 configured to capture target audio anda noise capturing device 104 configured to capture noise. While thetarget audio capturing device 102 and noise capturing device 104 areshown as distinct elements, in some embodiments they may be part of asingle element. Embodiments of the target audio capturing device 102 andnoise capturing device 104 will be described in more detail below.

The system 100 may include a sensor module 106. The sensor module 106may include various sensors configured to sense information about thesystem, including information related to the target audio source, thenoise source and/or the UAV itself. For example, the sensor module 106may include a GPS sensor 112 configured to sense the GPS location of thesystem 100 (and therefore also the GPS location of the UAV 200 to whichthe system 100 is attached). The sensor module may also include sensorsconfigured to sense the relative direction of the target audio capturingdevice 104 and/or the noise capturing device 106. Other possible sensorswill be described in more detail below. The system 118 may include animage capturing device 118. The image capturing device 118 may be usedfor applications of the system where image capture is required (forexample, for filming purposes). The image capturing device 118 mayinclude a video camera suitable for the type of filming required (forexample, for recreational use the camera may be a relatively light andinexpensive video camera whereas for cinematic filming, the camera maybe a relatively heavy, but higher quality, cinematic camera). The imagecapturing device 118 may also include a photographic camera forcapturing still images, which may be more suitable in some applicationswhen video is not required. At least some of the components of thesystem may be mounted on a gimbal (not shown in FIG. 1). In oneembodiment, the whole system 100 may be mounted to the UAV 200 on agimbal such that the system is steerable with respect to the UAV 200. Inother embodiments, parts of the system may be mounted on a gimbal,attached to the rest of the system or to the UAV itself. For example,the target audio capturing device 102, noise capturing device 104 andimage capturing device 118 may together, or each be mounted on a gimbal,in turn attached to the system 100. In this way, these devices may besteerable with respect to the rest of the system 100, with respect tothe UAV 200 and/or with respect to each other.

The system 100 may include a control module 114. The control module 114may be connected to an actuator 116. The actuator 116 may be a gimbalmotor or motors, which may be controlled to steer the gimbal or gimbalsdescribed above. For example, where the whole system is mounted on agimbal, the actuator may be controlled to steer the gimbal so that thetarget audio capturing device 102 is directed towards the target audiosource. In another example, if the target audio capturing device 102 ismounted on a first gimbal and the image capturing device 118 is mountedon a second gimbal, the actuator may include a first gimbal motor, whichmay be controlled to steer the first gimbal so that the target audiocapturing device 102 is directed towards the target audio source, and asecond gimbal motor, which may be controlled to steer the second gimbalso that the image capturing device 102 is directed towards the imageneeding to be captured.

The modules and devices described above may all be connected to aprocessing unit 108. The processing unit 108 may be configured toreceive inputs from the various modules and devices, to processinformation, and to produce outputs that control the operation of thevarious modules and devices. For simplicity, the processing unit 108 isshown in FIG. 1 as a single module, but it may be divided into multiplemodules, some of which may be incorporated into the other modules of thesystem. For example, the sensor module may have the capacity toundertake some processing of information itself, in which case theprocessing unit may be considered to be, at least in part, incorporatedinto the sensor module.

The system may also include a communication module 110. Thecommunication module may be configured for two-way communication with aremote processing unit 120. Such two-way communication may be by anysuitable wired or wireless communication protocol. While the remoteprocessing unit 120 of FIG. 1 is shown as being part of the UAV 200, insome embodiments the remote processing unit may be located remote fromthe UAV 200 (for example, a laptop located on the ground). In this way,the processing of the methods described below may be processed theprocessing unit 108, by the remote processing unit 120 or a combinationof both. The communication module may communicate with a furthercommunication module (not shown) incorporated in the UAV. The furthercommunication module may be configured to communicate with a remotedevice (for example, a laptop that a user is using to control theoperation of the UAV). In this way, the system 100 does not need toestablish a separate line of communication with a remote device, but canrely on the existing line of communication between the UAV and remotedevice.

Various components of the system may be implement as one or moresuitable integrated circuits. For example, the integrated circuits maybe an ASIC or FPGA, which may be well suited for UAV applications due totheir relatively low weight, size and low power consumption.

The system 100 also includes a power source 122. The power sourcesupplies power to the various modules and devices of the system 100. Thepower source may be a battery, which may be replaced or recharged. Whilethe power source 122 is shown as part of system 100, in anotherembodiment, the power source may be part of the UAV 200. However, it maybe beneficial to provide a distinct power source for the system 100(rather than rely on the existing power source of the UAV), as the powersource of the UAV may be ‘noisy’, which could impact the quality ofsignals produced by the various modules and devices and ultimatelyimpact the quality of the noise filtering.

FIG. 2 is a block diagram showing the UAV 200 and system 100 describedabove in an example context. FIG. 2 shows the system 100 configured as apayload 210 mounted to a UAV 200. FIG. 2 shows a user 201 who iscontrolling the UAV 200 and using the system 100 for capturing targetaudio from a target audio source 203. For example, the user may be usingthe UAV 200 to film a protest, where it is desirable to capture what theprotesters are saying. The user 201 uses a remote device including aremote processing unit 120, which is in wireless communication with thecommunication module to control the system 100. The user 201 may alsouse the remote processing unit 120 to control the UAV 200 (for example,to fly the UAV).

The payload 210 is mounted to the UAV 200 via a gimbal, and may bemovable independent from the UAV 200. The user may fly the UAV 200towards the target audio source. The user controls the system 100 sothat the image capturing device (not shown) is directed toward thetarget audio source (thereby filming the scene). The target audiocapturing device 102, which has been configured to be aligned with theaxis of the image capturing device, will therefore also capture targetaudio from the target audio source 203.

Noise is captured from one or more noise sources 205 a 205 b (such asthe propellers of the UAV 200) by a noise capturing device 104. As shownin FIG. 2, the noise capturing device 104 may have two parts 104 a 104b. Each part may be positioned on or about the UAV 200 to better capturethe noise from a noise source (for example, they may be mounted eitherside of the UAV 200 to capture noise from the left 205 a and right 205 bpropellers) and they may be directed towards the noise source. In thisembodiment, while the noise capturing device 104 may be considered partof the system 100, it is mounted directly onto the UAV 200 rather thanas part of the payload 210. In this way, when the user steers thepayload (for example, to direct the image capturing device towards ascene being filmed), the position and direction of the noise capturingdevice 104 with respect to the noise source 205 a 205 b is not affected.

As will be described in more detail below, the processing unit of thesystem 100 uses the sensor module to determine the position and/ordirection of the target audio capturing device 102 relative to the noisecapturing device 104. The parameters of a noise filtering algorithm areadjusted using this position and/or direction information. Theprocessing unit uses the adjusted noise filtering algorithm to outputfiltered target audio based on the captured target audio. Thus thesystem is able to capture reasonably clean target audio from the targetaudio source 203, minimizing interference from the noise of the UAV 200itself. The filtered target audio is time-synced with the capturedvideo. The filtered target audio and video may be livestreamed to theremote processing unit 120 and may be viewable by the user 201.

Target Audio and Noise Capturing Devices

In one embodiment, the target audio capturing device 102 and noisecapturing device 104 may be a sound capturing device. The soundcapturing device may include any suitable number of microphones. Withoutlimitation, such microphones may be MEMS microphones, condensermicrophones (for example, electret condenser microphones), electretmicrophones, parabolic microphones, dynamic microphones, ribbonmicrophones, carbon microphones, piezoelectric microphones, fiber opticmicrophones, laser microphones and/or liquid microphones. A microphonemay be used because of its particular polar pattern, for example, ahyper-cardioid shotgun microphone, a three cardioid microphone and/or anomnidirectional microphone. The microphone may be formed as an array.For example, an array of two or three cardioid or omnidirectionalmicrophones, or an array of MEMS microphones.

The microphone may be selected to take advantage of its particularproperties. Such properties may include directionality (as shown by itscharacteristic polar pattern), frequency response (which may correspondto the target audio and/or noise), or signal to noise ratio.

Using directional beaming forming (MVDR or similar) may requirecharacterisation by impulse response to generate the beam. The beamperforms optimally when the sound source or noise source is exactly inthe direction characterized for, but performance degrades as the array'sphysical orientation changes and the sources are no longer aligned. Inother words each UAV would ideally need to be initially calibrated witha wide range of relative gimbal orientations. In use a specific gimballocation would be approximated to the nearest calibration point, withvarying degrees of effectiveness.

In one embodiment, the sound capturing device may be an array of MEMSmicrophones, that have been configured to have a polar pattern as shownin FIG. 3a . The polar pattern illustrates that the sound capturingdevice 311 has two lobes of sensitivity. As will be described in moredetail below, the target audio and noise may then be used for noisefiltering to produce filtered target audio.

This sound capturing device 311 is advantageous because the polarpattern is such that if the first lobe 313 of sensitivity is directedtowards the target audio source 303 the sound capturing device willcapture the target audio. The shape of the second lobe 315 ofsensitivity means that the sound capturing device will also be able tocapture noise from noise sources (for example propellers 305 a 305 b,which are fixed in relation to the location of the sound capturingdevice 311) outside the target audio. Another advantage is thatregardless of the relative position of the first lobe 313 of sensitivitywith respect to the noise source, the noise from the noise source isgoing to be captured within the second lobe 315 and with reasonablyconsistent gain. This is illustrated by FIG. 3b . The target audiosource 303 has moved relative to the sound capturing device (bycomparison to FIG. 3a ), requiring the sound capturing device 311 to beredirected so that the first lobe 313 of sensitivity is directed towardsthe target audio source 303. However, noise from the noise sources 305 a305 b, while now a different direction relative to the sound capturingdevice, will still be captured by the second lobe 315. Further benefitsof this will become apparent when the method for noise filtering isdescribed in more detail below.

A beam with a wide capture area with near-equal gain across frequenciesmay be more suitable for mounting on moving gimbal systems on UAVs, asthe inaccuracies of the response to the relative change in position ofthe noise source(s) are reduced. For the purposes of noise filteringusing beamforming to capture the sound and noise sources, the wider beamis advantageous as it will provide a more complete capture of the noisebeing received by the audio capture device. The lesser variation ofresponse across the beam capture arc also helps by reducing the error insource separation, which is worse the further the response strays fromthe target.

The array and beamformer may use a null beam with the wide and invariantresponse that closely approximates the ideal situation described above.This allows its use in gimbal mounted drone systems that will move thearray relative to noise sources while maintaining performance requiredfor noise filtering.

This also means that a single implementation with the wider null beam istransferable between drone frames with distinct motor positions whilestill capturing them as noise sources. It may allow not just transferbetween drone frames of the same model but also between drones ofdifferent model.

In one example the null beam may be wide enough to capture all of thenoise sources. In that case the noises source such as the motors/rotorsmay be defined in terms od known relative directions, known relativedistance the audio recording device (quite close) and of a signal powersignificant compared to the signal of interest, perhaps ranging from −5dB down to +10 dB up from the signal. For example, in a quad copter UAV,depending on the location of the gimbal, the null beam may be at least180°. In other examples, with different main beam widths, the null beammay be 355°, 350°, 340°, 330°, 320°, 310°, 300°, 290°, 280°, 270°, 260°,250°, 240°, 230°, 220°, 210°, 200°, or 190°. Alternatively the null beammay be defined by the absence of the main beam. For example if the userselects a main beam width of X°, then the null beam width may be 360−X°,or may be 360−X°−Y°, where Y is a buffer width, which may be fixed ormay be user selectable, or may be determined based on an algorithm.

The level of reasonably consistent gain across the angle of the nullbeam may vary depending on the application. For example, in a typicalcommercial audio capture on a quad copter UAV, the gain may vary by lessthan 20% across the beam width. In other examples, the null beam gainmay vary by less than 40, 35, 30 or 25% across the beam width.

The MEMS array may use an end-fire line microphone array as the audiocapture device. The end-fire line array would have a form factor similarto a standard shotgun microphone. This shape would be more suitable thanother arbitrary array structures which may be less suitable for themounting constraints of a drone gimbal.

The end-fire array structure also lends itself to having the greatestpossible directivity on either end of the array which allows for thebest capture of the signal of interest/noise source with high rejectionin other directions.

Such an array may provide the additional advantage of a wider and moreeffective frequency response. For example, the frequency performance maybe characterised by the level of variation in gain across beam width ofthe null beam. The level of reasonably consistent gain across thefrequency range may vary depending on the application. For example, thelowest useable frequency may have a null beam gain which varies by lessthan 20% across the beam width, such as at 150 Hz or 1 kHZ. In a typicalcommercial audio capture on a quad copter UAV, this may result in auseable frequency range of 150 Hz to 20 kHz for noise filteringpurposes.

In another embodiment, beamforming may be used to define lobes ofsensitivity that capture both the target audio source and noise source.For example, referring to FIG. 4a , there is shown a polar pattern for asound capturing device 411 following suitable beamforming. A first beam,as shown by a first lobe of sensitivity 413, has been configured tocapture target audio from a target audio source 403. Second beams, asshown by second lobes of sensitivity 415 a and 415 b, have beenconfigured to capture noise from noise sources (for example propellers405 a 405 b, which are fixed in relation to the location of the soundcapturing device 411). So formed, the sound capturing device 411 is ableto capture target audio and noise, which may then be used for noisefiltering to produce filtered target audio as described below.

If the target audio source moves relative to the sound capturing device,the same beams will not be effective. This is illustrated by FIG. 4b .In this figure, the target audio source 403 has moved (by comparison toFIG. 4a ), requiring the sound capturing device 411 to be steered sothat the first lobe 413 of sensitivity is directed towards the targetaudio source 403. However, having moved, the noise from the noisesources 305 a 305 b will no longer be captured by the existing lobes ofsensitivity (as shown by dashed lines 415 a 415 b), thereby limiting thenoise that is captured. Therefore, in order to more accurately capturenoise, a new beamforming configuration is implemented, as shown by 417 aand 417 b. Such beamforming may be understood as steering the soundcapturing device in that it redirects the beams. In another embodiment,rather than physically redirecting the sound capturing device, if thetarget audio source is no longer within the first lobe of sensitivity,then a new beamforming configuration may be implemented where the secondlobes of sensitivity remain unchanged, but the first lobe is steered soas to capture the target audio source. This approach may be beneficialas there is no need to physically redirect the sound capturing device,instead relying on steering the beams.

The sound capturing device may be enabled with multiple beamformingconfigurations. The sound capturing device may be configured toimplement a suitable beamforming configuration depending on the relativelocation of the target audio source with respect to the noise source(s).In order that the appropriate beamforming configuration is implementedit may be necessary to detect the relative directions of the targetaudio source with respect to the noise source. For UAVs, since theprimary noise source (namely the motor and propeller assembly) is fixedwith respect to the UAV, it is only necessary to detect the direction ofthe target audio source with respect to the UAV.

In another embodiment, the target audio capturing device may be distinctfrom the noise capturing device. The target audio capturing device maybe one or more microphones. Without limitation, such microphones may beMEMS microphones, condenser microphones (for example, electret condensermicrophones), electret microphones, parabolic microphones, dynamicmicrophones, ribbon microphones, carbon microphones, piezoelectricmicrophones, fiber optic microphones, laser microphones and/or liquidmicrophones. A microphone may be used because of its particular polarpattern, for example, a hyper-cardioid shotgun microphone, athree-cardioid microphone and/or an omnidirectional microphone. Themicrophone may be formed as an array. For example, an array of two orthree cardioid or omnidirectional microphones, or an array of MEMSmicrophones. The microphone may be selected to take advantage of itsparticular properties suitable for target audio capture. Such propertiesmay include directionality (as shown by its characteristic polarpattern), frequency response (which may correspond to the target audio),or signal to noise ratio.

If multiple separate microphones are used, the microphones may beuniformly distributed about a UAV, and may be selectively turned on tocapture target audio from a particular direction. In other embodiments,an array of target audio capturing devices may be uniformly radiallyspaced from each other on the UAV and selectively activated to capturetarget audio from different directions.

The noise capturing device may be one or more microphones. Withoutlimitation, such microphones may be MEMS microphones, condensermicrophones (for example, electret condenser microphones), electretmicrophones, parabolic microphones, dynamic microphones, ribbonmicrophones, carbon microphones, piezoelectric microphones, fiber opticmicrophones, laser microphones and/or liquid microphones. A microphonemay be used because of its particular polar pattern, for example, ahyper-cardioid shotgun microphone, a three-cardioid microphone and/or anomnidirectional microphone. The microphone may be formed as an array.For example, an array of two or three cardioid or omnidirectionalmicrophones, or an array of MEMS microphones. The microphone may beselected to take advantage of its particular properties suitable fornoise capture. Such properties may include directionality (as shown byits characteristic polar pattern), frequency response (which maycorrespond to the target audio), or signal to noise ratio. The targetaudio capturing device may be the same type of microphone as the noisecapturing device or may be a different type.

Position

The sound capturing device may be located as part of the payload (forexample, 210 of FIG. 2) mounted to the underside of the UAV (which maybe via a gimbal). The sound capturing device may be connected to the UAVby a gimbal. In this way, the sound capturing device may be steerablewith respect to the UAV.

As shown in FIG. 5, the sound capturing device 507 may be mounted to theUAV 500 in the space that is within 10 degrees of the plane of the motorand propeller assembly 509. This is advantageous as the noise from themotor and propeller assembly 509 is at a minimum in this space. Thesound capturing device may be mounted towards the front or the back ofthe UAV (rather than the side) to maintain balance. The sound capturingdevice (or the gimbal to which it is attached) may be mounted via aconnection configured to isolate and/or dampen vibrations produced bythe UAV.

Where there are distinct target audio capturing devices and noisecapturing devices, these may be distanced from each other so as tominimize noise picked up by the target audio capturing devices. Forexample, the target audio capturing device may be positioned on a UAVpayload hanging below a UAV and angled to face the ground, while thenoise capturing device may be mounted closer to and directed towards thenoise source(s). In another example, the target audio capturing devicemay be positioned directly on the side of the UAV within 10 degrees ofthe plane of the motor and propeller assembly (similar to thearrangement described in relation to FIG. 5. The target audio capturingdevice (or the gimbal to which it is attached) may be mounted via aconnection configured to isolate vibrations.

The noise capturing devices may be fixed to or movable relative to thepayload or the UAV. The noise capturing devices are configured to facenoise sources, which are to be filtered out from target audio. Examplesof noise include, but are not limited to, noise from a UAV motor and/orpropeller assembly or wind noise. The noise capturing devices may belocated on the arms of a UAV or near other sources of noise on the UAV.

Movability

The sound capturing device (or target audio capturing device, ifdistinct from the noise capturing device) may be movable relative to theUAV (and/or to the noise capturing device). For example, they may bemounted via an independently steerable gimbal. In the case that soundcapturing device is not movable with respect to the UAV, the UAV itselfmay be moved to direct the sound capturing device towards a target audiosource.

In one case, the sound capturing device (or target audio capturingdevice, if distinct from the noise capturing device) may be aligned withan image capturing device, such that the sound capturing device isdirected towards the target audio source which is also being captured bythe image capturing device. In one embodiment, the sound capturingdevice and image capturing device may be mounted on the same gimbal toensure they remain in alignment regardless of the direction they face.

Sensor Module

The sensor module 106 (introduced in relation to the system 100 ofFIG. 1) payload may include target sensors for sensing data about atarget audio source. Examples of target sensors include but are notlimited to vision sensors (for example, imaging devices capable ofdetecting visible, infrared, or ultraviolet light, such as cameras orthermographic cameras), proximity sensors (for example, ultrasonicsensors, lidar, laser rangefinders, time-of-flight cameras) or otherfield sensors (e.g., magnetometers, electromagnetic sensors). Suchvision sensors may be in addition to, or instead of, the image capturingdevice previously described. Sensor data may be passed to the processingunit 108 or communicated to a remote device.

Target data from the target sensors may be used to control the directionof the target audio capturing device to track the target audio source.The processing unit may be configured to track the target audio sourceautomatically (for example, the target audio source may include a radiotransceiver which the processing unit, via a suitable transceiverincluded in the sensor module, is able to detect the location of) or thetracking may require some input from a user (for example, a user maytrack the target audio source visually via a video captured by the imagecapturing device or a suitable vision sensor, and manually redirect thetarget audio capturing device).

Where the target audio capturing device is attached via a gimbal, thetarget audio source may be steered independently from the payload and/orUAV. Where the payload is attached via a gimbal, the payload itself maybe steered (and thereby redirect the target audio capturing device). Thesensor module may include gimbal sensors to detect the orientation ofthe gimbal, thereby allowing the relative direction of the target audiosource or target audio capturing device with respect to the UAV to bedetermined. For example, if a user manually controls the payload totrack a target audio source via a video feed, the gimbal sensors maysense the orientation of the gimbal and therefore the relative directionof the target audio source with respect to the UAV can be determined.

The data from the sensors may be may be associated with target audiosource data, and may be used to obtain measurements from a target, tomap an area, or assist UAV navigation. Sensor data may be live streamedto a remote location, communicated to a remote device, or locallystored.

The sensor module may include other sensors to determine position,orientation and/or movement of the UAV, the payload, and/or the targetaudio capturing device and noise capturing device. Examples of sensorsinclude, but are not limited to, location sensors (for example, GPSsensors or mobile device transmitters enabling location triangulation),inertial sensors (for example, accelerometers, gyroscopes or inertialmeasurement units), altitude sensors, and/or pressure sensors. Forexample, the sensor module may include an electronic compass formeasuring the orientation (for instance azimuth and inclination) of theUAV.

While the sensors above are described as part of the sensor module thatis part of the system 100 mounted on the payload attached to the UAV.Some may be mounted on or incorporated in the UAV itself. For example,the UAV may already include a GPS sensor and the system 100 may beconfigured to receive data from the existing GPS system via thecommunication module.

“Directional data” as determined by the sensors, may include therelative angle of the sound capture device to the drone/noise sources.This may be derived from the telemetry data from the rotating gimbal onwhich the device is mounted, or other sensors, for selecting appropriateinput parameters in a spatial noise filtering system as described below.

Target Audio Source Location

Sensor data (including, for example, gimbal data and GPS data) may becombined to calculate the absolute location of a target audio source.For example, a user may remotely direct the target audio capturingdevice towards the target audio source. A rangefinder (for example, alaser rangefinder, aligned with the direction of the target audiocapturing device) may calculate the distance between the UAV payload andthe target audio source. The gimbal sensor may detect the relativedirection of the target audio capturing device with respect to the UAVand an accelerometer may be used to detect the orientation of the UAV.If the relative direction of and distance to the target audio source isknown, and the absolute location of the UAV is known (for example usingGPS), then the absolute location of the target audio source can bedetermined.

Payload

As described in relation to FIG. 2, the payload of the UAV may belocated under the UAV. The payload may include the system for noisefiltering described herein. Some aspects of the system however may beshared with the UAV (for example, a power source).

The payload may be removably or permanently attached to the UAV. Thepayload may be attached via a gimbal, enabling the payload to be steeredindependently from the UAV (i.e. by controlling the yaw, roll and pitchof the gimbal and therefore the payload).

Processing

The system 100 may partially or completely process audio and noise dataonboard to produce filtered target audio. Alternatively, the system 100may store audio and noise data for post processing. The system 100 mayadditionally include a data storage component which stores datacollected and/or processed by the processing unit 108. In oneembodiment, the data storage component may store data when connectivityis lost between the system 100 and a remote processing unit 108, fortransmission at a later time when connectivity is restored. The datastorage component may be an SD card.

Characteristics of UAV

The system for noise filtering 100 may be combined with other systemsand methods which may reduce noise produced by the UAV itself. Forexample, the UAV's motor and propeller assembly may be shrouded, the UAVmay include noise-absorbing material, and/or the UAV may be providedwith noise-cancelling emitters.

The UAV may navigate by remote control, pre-programming, or autonomousnavigation. The UAV may include one or more propulsion units, allowingthe UAV to move with up to six degrees of freedom. The UAV may be anysuitable size. The UAV may include a central body, and one or more armsextending outwardly from the central body including the UAV propulsionunits. The central body may include a housing, including UAVelectronics.

The UAV may itself include one or more sensors. Examples of sensors ofthe UAV may include, but are not limited to, location sensors (forexample, GPS sensors, mobile device transmitters enabling locationtriangulation), vision sensors (for example, imaging devices capable ofdetecting visible, infrared, or ultraviolet light, such as cameras),proximity sensors (for example, ultrasonic sensors, lidars,time-of-flight cameras), inertial sensors (for example, accelerometers,gyroscopes or inertial measurement units), altitude sensors, pressuresensors, audio sensors (for example, microphones), and/or field sensors(for example, magnetometers or electromagnetic sensors).

A battery may be coupled to the UAV 200. The battery may be coupled to aUAV to provide power to one or more components of the UAV. The batterymay provide power to one or more propulsion units and any othercomponent of the UAV while coupled to the UAV. In some embodiments, thebattery may also provide power to the system for noise filteringincluding the target audio capturing device. In other embodiments, thesystem relies on its own power source.

While described in relation to a UAV, embodiments may be used to improvetarget audio source audio capture in any suitable moving vehicles,including but not limited to UAVs, helicopters, rotorcraft, verticallift systems and fixed-wing aircraft.

Target Audio and Noise

The UAV may be configured to fly in any suitable environment for targetaudio capture, including both indoor and outdoor environments. Targetaudio may be ambient audio, sound produced by humans, animals, machines,the environment or any other audio which one may wish to capture.

The distance between the UAV and the target audio source may varyaccording to the application.

Noise captured by noise capturing device may include noise produced bythe UAV itself or ambient noise. Examples of noise produced by the UAVinclude noise produced by the motor and propeller assembly, noiseproduced by onboard instruments (such as gimbal motors or cameras) ornoise produced by interaction of the AUV with airflow. Ambient noise mayinclude general wind noise, noise from surrounding airborne vehicles orother environmental noise, such as traffic noise.

In some embodiments the target audio source may be closer to the targetaudio capturing device than the noise capturing device. Alternatively,the noise source may be closer to the target audio capturing device thanthe target audio source.

Remote Processing Unit

The remote processing unit 120 may be incorporated in any suitableremote device including but not limited to a personal computer, laptop,smartphone, tablet or custom device. The remote device may include auser interface for controlling the UAV and/or system for noise filtering100, and a display for displaying data from the UAV and/or system. Suchdata may include sensor data and/or target audio or noise data.

The system 100 may include a control mechanism for starting and stoppingaudio capture. This may be useful, for example, in live broadcasting. Auser having a remote device may communicate with the system 100 to startaudio and/or video capture, redirect the target audio capturing deviceor image capturing device, and stop audio and/or video capture. The usermay selectively capture audio alone or video alone.

In some embodiments, the payload may include a speaker allowing remotecommunication. The remote device captures an audio message from the user(for example, instructions for a person receiving a package) andtransmits the audio message wirelessly to the communication module ofthe system, which is then emitted by the speaker. In this way, remotecommunication is achieved.

Filtering Method

Having described the system and devices, various methods for noisefiltering will now be described. FIG. 6 shows a method of producingfiltered target audio using the system 100 for noise filtering describedabove. Such steps may be carried out by the processing unit or theremote processing unit, or a combination of both.

At step 602, the direction of a target audio source relative to thesystem is detected. In one embodiment, the target audio source mayinclude a radio transceiver which communicates its position to thesystem 100, from which the direction towards the target audio source canbe detected. In another embodiment, a user may use a video feed to steeran image capturing device to the target audio source by ensuring thetarget audio source is within the field of view of the image capturingdevice. For example, the image capturing device may be mounted to theUAV via a gimbal that can be controlled so that the field of view of theimage capturing device faces the target audio source. In anotherexample, the image capturing device may be attached to the UAV, and sothe user may move the UAV (by flying it to a certain position) so thatthe image capturing device faces the target audio source. By determiningthe relative direction of the image capturing device with respect to thesystem, it is possible to detect the direction of the target audiosource.

At step 604, the target audio capturing device (or the sound capturingdevice, in embodiments where the target audio capturing device and noisecapturing device are provided in the same device) is directed towardsthe target audio source. In embodiments where an imaging capturingdevice has been used to detect the direction of the target audio source,the target audio capturing device may be aligned with the imagecapturing device such that it is automatically directed towards thetarget audio source. In other embodiments, the target audio capturingdevice may be redirected towards the target audio source, for example bycontrolling the gimbal to which the target audio capturing device isattached.

At step 606, the noise capturing device is directed towards the noisesource. Where the primary noise source is the noise from the UAV's motoror propeller assembly, the noise capturing device or devices may alreadybe directed to the noise source.

At step 608, the relative directions between the target audio capturingdevice and noise capturing device(s) is determined. Since the relativedirection of the target audio capturing device with respect to thesystem is known (being the same as the relative direction of the targetaudio source detected at step 602) and the direction of the noisecapturing devices is known, then the relative direction between thetarget audio capturing device and noise capturing device(s) isdetermined.

At step 610, target audio from the target audio source is captured usingthe target audio capturing device (or the sound capturing device, inembodiments where the target audio capturing device and noise capturingdevice are provided in the same device). Noise is captured from thenoise source using at least one noise capturing device (or the soundcapturing device, in embodiments where the target audio capturing deviceand noise capturing device are provided in the same device).

At step 612, the parameters of a noise filtering algorithm are adjustedusing the directional data obtained at step 608.

At step 614, filtered target audio is produced using the adjusted noisefiltering algorithm.

In order that target audio is continually captured, the method maycontinually or periodically repeat steps 602-608 in case the targetaudio source moves with respect to the system (for example, the targetaudio source may be mobile, or the UAV may move with respect to thetarget audio source).

FIG. 7 shows another embodiment of a method which relies on beamforming.

At step 702, the relative direction of a target audio source relative tothe system is detected in much the same way as described in relation tostep 602.

At step 703, the relative direction of a noise source relative to thesystem is detected.

Where the primary noise source is the noise from the UAV's motor orpropeller assembly, the relative direction will be known.

At step 705, the sound capturing device will be implemented with asuitable beaming configuration such that the beams are directed towardsthe target audio source and noise source.

At step 708, the relative directions between the target audio source andnoise source is determined.

At step 710, target audio from the target audio source is captured usingthe sound capturing device and noise is captured from the noise sourceusing the sound capturing device.

At step 712, the parameters of a noise filtering algorithm are adjustedusing the directional data obtained at step 708.

At step 714, filtered target audio is produced using the adjusted noisefiltering algorithm.

In order that target audio is continually captured, the method maycontinually or periodically repeat steps 702-708 in case the targetaudio source moves with respect to the system.

FIG. 8 shows another embodiment of a method which relies on beamforming.

At step 801, the sound capturing device is implemented with a firstbeamforming configuration.

At step 802, the relative direction of a target audio source relative tothe system is detected in much the same way as described in relation tostep 602.

At step 803, the relative direction of a noise source relative to thesystem is detected. Where the primary noise source is the noise from theUAV's motor or propeller assembly, the relative direction will be known.

At step 804, the sound capturing device is directed such that a targetaudio capturing beam is directed towards the target audio source.

At step 805, the sound capturing device will be implemented with asuitable beaming configuration such that the beams are directed towardsthe target audio source and noise source.

At step 808, the relative directions between the target audio source andnoise source is determined.

At step 810, target audio from the target audio source is captured usingthe sound capturing device and noise is captured from the noise sourceusing the sound capturing device.

At step 812, the parameters of a noise filtering algorithm are adjustedusing the directional data obtained at step 808.

At step 814, filtered target audio is produced using the adjusted noisefiltering algorithm.

In order that target audio is continually captured, the method maycontinually or periodically repeat steps 802-808 in case the targetaudio source moves with respect to the system.

FIG. 9 shows a schematic diagram of a method for producing filteredtarget audio Z(t) using a sound capturing device according to oneembodiment. The sound capturing device includes an array of microphones(denoted 1, 2, . . . M), which each capture sound data in the timedomain X₁(t), X₂(t), . . . X_(M)(t). A Fourier transform is used tochange the domain of the sound data to the frequency domain X₁(ω),X₂(ω), . . . X_(M)(ω).

The sound data X₁(ω), X₂(ω), . . . X_(M)(ω) is passed to Beamformer 0,which uses the directional data (for example, the directional datadetected at step 702 or step 802, described above) to apply a suitablebeamforming configuration so that the resulting target audio beam Y₀(ω)is directed towards the target audio source.

The sound data X₁(ω), X₂(ω), . . . X_(M)(ω) is also passed tobeamformers n, which use the directional data (for example, thedirectional data detected at step 703 or step 803, described above) toapply a suitable beamforming configuration so that the resulting noisebeam(s) Y_(n)(ω) is directed towards the noise source(s).

The target audio beam Y₀(ω) and noise beam Y_(n)(ω) are provided to asquare law unit which calculates the energy magnitude per frequency binfor each beam. The resulting data is supplied to a PSD Estimation unitwhich estimates the PSD for each beam. This may be done using the Welchmethod. The Welch method relies on directivity data. The directivitydata may be precalculated from impulse response system characterisation.The PSD Estimation unit uses directional data to select the appropriatedata when estimating the PSD for each beam.

The PSD Estimation units produces weights, which are supplied to asuitable filter (such as a Wiener filter, as shown in FIG. 9), whichproduces filter H(ω) that is applied to the target audio beam Y₀(ω). Aninverse Fourier transform converts to the time domain, producing thefiltered target audio Z(t).

While the sound capturing device will continually capture sound data X₁(t), X₂(t), . . . X_(M)(t), as the relative direction of the targetsource with respect to the noise changes (for example, due to a movingtarget source), new beamforming configurations and PSD estimations areapplied, thereby improving the filtered target audio Z(t).

In embodiments where the sound capturing device is physically steerablewith respect to the UAV, once Beamformer 0 has been applied, it may notbe necessary to reconfigure the target audio beam. If the relativedirection of the target audio source changes, the sound capturing devicewill be redirected so that the target audio beam continues to capturetarget audio. However, as the relative direction of the noise source(s)will have changed, new noise beam(s) will need to be implemented byBeamformer n.

FIG. 10 shows a schematic diagram of a method for producing filteredtarget audio Z(t) using a sound capturing device where the soundcapturing device has the polar pattern described in relation to FIGS. 3aand 3b . Beamformer 0 produces the target audio beam (corresponding tothe first lobe of sensitivity) and Beamformers n producer the noise beamn (corresponding to the second lobe of sensitivity). As the relativedirection of the target audio source changes (for example, the targetaudio source moves), the direction of the sound capturing device ischanged (for example, it may be mounted via a gimbal allowing it to besteered). The beams themselves do not need to change since the secondlobe of sensitivity will capture the noise sources regardless of thedirection. Also, since the noise beam has approximately uniform gain,the energy capture by the noise beam is largely immune to the relativeposition of the noise source. Therefore, using the array of FIGS. 3a and3b may be simpler as it does not need to be regularly updated with newdirectional data.

In this case “directional data” may mean the spatial relationships ofeach element in the array forming the sound capture device to the noisesources. This may be used to calculate the beamformers for capturing thesound and noise sources prior to using the capture device.

Many noise filtering processes are based around having estimations ofthe noise mixed into the input audio, to filter only that noise out.This means the accuracy of that estimation is key to the actualperformance of these processes.

In one instance of a noise filtering system that can be used with agimbal-mounted microphone array, the system may receive spatiallyseparated target sound and noise sources via 2 separate beamformerspointed towards those sources. While the beams themselves enact spatialfiltering there is still leakage of non-target sound level into eachbeam. It is possible to use the known responses of each of the beams inknown target directions of interest to obtain an estimation of whatcomponents of audio have originated from those directions:

PSD_(sources)(f,t)=G ⁻¹(f)×PSD_(beams)(f,t)

where PSD_(beams) is the power spectral density (PSD)_ of the audiocaptured by each beam pointing in target sound and noise sources ofinterest, G⁻¹ is the inversed square matrix containing the gain of eachbeamformer in each of the directions of interest, and PSD_(sources) isthe estimated PSD of each of the different sources.

When the diagonal elements of the gain matrix G are small, the inverseoperation performed on digital systems are prone to precision-levelerror, potentially degrading performance greatly. The severity of thiserror rises with the size of the gain matrix G, which gets larger withincreasing number of beams used. This error can be alleviated withregularization:

G _(regularized) =G+R×I

Where I is the identity matrix and R is the regularization factor. R istypically a small, arbitrary number selected to ensure the diagonalelements are not too small. However, regularization causes the matrix tono longer accurately represent the gain of the beamformers, so theperformance of the noise filtering is still degraded accordingly.Generally, to meaningfully capture the noise sources relevant in a UAVsystem with narrow beams both ‘sides’ of motors must have their owndedicated beams in addition to the target sound source beam. In UAVframes with motors that are spatially located further apart, more beamsmay be needed, and the greater the degree of numerical error introduced.

By utilizing the wide null beam implementation which captures allnon-target sources in one beam with near-invariant gain we can model allthe noise input from this capture area as a single ‘noise source’ fromone direction. In this model there is only two beams that capture twodirections of interest, one of the target sound source and one of the‘noise source’. The G matrix that consists of the directivity of eachbeamformer in each signal direction of interest is thus reduced to a 2×2matrix, minimizing the odds of precision error and also reducing theamount of regularization that needs to be done, if any. This mayincrease the accuracy of the resulting noise filtered output.

This reduction of the problem space also means the calculation load isreduced, making it more feasible in a real-time implementation on theUAV where power consumption is a significant design consideration.

Although the response is relatively angle- and frequency-invariant forthe wider null beam for large angles, there is still a small degree ofdeviation from the ideal unity gain and it may improve the performanceof noise filtering if the true response at a particular angle related tothe primary noise source(s) is used. The telemetry of the drone and thegimbal may be used in some applications to select the appropriate gainsto use in the G matrix for the current relative angle of the soundcapture system to noise sources.

FIG. 11 shows a schematic diagram of a method for producing filteredtarget audio Z(t) using distinct noise capturing devices (N1, N2) and atarget audio capturing device (S). In this embodiment, there is nobeamforming.

The target audio from the target audio capturing device is passedthrough a Fourier transform unit, to produce a target audio data in thefrequency domain X_(S)(ω). The noise signal from noise capturing devicesN1 and N2 is also passed through a Fourier transform unit, to producenoise audio data X_(N1)(ω) and X_(N2)(ω).

These are then passed through a Square Law unit, PSD Estimation unit andWiener filter in much the same way as described in relation to FIG. 9,producing filtered target audio X(t).

The target audio capturing device may be steerable with respect to theUAV while the noise capturing devices may be fixed (for example, theymay be permanently directed towards the motor and propeller assembly).Therefore the relative direction of the target audio signal may change,which information is supplied to the PSD Estimation unit.

FIG. 12 shows a diagram of noise filtering according to one embodiment.This shows noise from a noise source being filtered out from targetaudio using an adaptive filter to produce a filtered target audiooutput.

FIG. 13 shows a schematic diagram of noise filtering according to oneembodiment. Noise and target audio are captured. The signals arepre-amplified and then digitized. Then noise is filtered out from thetarget audio. The filtered target audio may be stored/transmitted indigital and/or analogue format.

While the present invention has been illustrated by the description ofthe embodiments thereof, and while the embodiments have been describedin detail, it is not the intention of the Applicant to restrict or inany way limit the scope of the appended claims to such detail.Additional advantages and modifications will readily appear to thoseskilled in the art. Therefore, the invention in its broader aspects isnot limited to the specific details, representative apparatus andmethod, and illustrative examples shown and described. Accordingly,departures may be made from such details without departure from thespirit or scope of the Applicant's general inventive concept.

1. A method for noise filtering including: receiving directional datacorresponding to the relative directions of at least one noise sourceand at least one target audio source; capturing noise data from the atleast one noise source; capturing target audio data from the at leastone target audio source; using the directional data to filter the noisedata from the target audio data; and outputting filtered target audio.2. The method as claimed in claim 1, wherein the directional data isused to determine at least one beamforming configuration wherein atleast one beam associated with the beamforming configuration captures atleast one of the noise source or the target audio source and the methodincludes the step of applying the beamforming configuration to sounddata captured by the sound capturing device.
 3. The method as claimed inclaim 2, wherein the noise data and target audio data is captured usinga sound capturing device, and the directional data is used to estimatethe power of the noise data and/or target audio data at the soundcapturing device.
 4. The method as claimed in claim 3, wherein the soundcapturing device includes a target audio capturing device and a noisecapturing device, and the directional data is used to estimate the powerof the noise data and/or target audio data at the target audio capturingdevice. 5.-16. (canceled)
 17. A method for noise filtering including:receiving directional data corresponding to the relative directions ofat least one noise source and at least one target audio source; steeringa sound capturing device to capture noise data from the at least onenoise source and to capture target audio data from the at least onetarget audio source; using the directional data to filter the noise datafrom the target audio data; and outputting filtered target audio. 18.The method as claimed in claim 17, wherein the step of steering thesound capturing device includes applying a beamforming configuration toredirect at least one beam to capture the at least one noise sourceand/or the at least one target audio source.
 19. The method as claimedin claim 17, wherein the sound capturing device is mounted via a gimbaland the step of steering the sound capturing device includes steeringthe gimbal to redirect the sound capturing device.
 20. The method asclaimed in claim 19, wherein the sound capturing device includes a noisecapturing device to capture the noise data and a target audio capturingdevice to capture the target audio, and wherein the target audiocapturing device is mounted via a gimbal and the step of steering thesound capturing device includes steering the gimbal to redirect thetarget audio capturing device towards the target audio source. 21.-27.(canceled)
 28. A method of noise filtering including: determining a beamforming pattern comprising a main beam and a null beam from a soundcapturing device mounted to a UAV; capturing noise data using the nullbeam; capturing target audio data using the main beam; filtering thenoise data from the target audio data; and outputting filtered targetaudio.
 29. The method of claim 28 wherein the null beam captures all ofthe significant noise sources.
 30. The method of claim 28 wherein thenull beam has a beam width of at least 180°.
 31. The method of claim 28wherein the null beam has a beam width of 360°−X°, where X is the beamwidth of the main beam.
 32. The method of claim 28 wherein the null beamhas a gain which varies by less than 20% across it's beam width.
 33. Themethod of claim 28 wherein the null beam has a frequency range, definedby a lower frequency, where the gain which varies by less than 20% atthe lower frequency.
 34. The method of claim 28 wherein filtering thenoise data from the target audio data uses a simplified 2×2 gain matrix,which characterises the gain of the main beam in the forward directionand reverse direction, and the null beam in the forward direction andreverse direction.
 35. A gimbalised microphone configured to use themethod of claim
 28. 36. The gimbalised microphone of claim 35, whereinthe microphone is a MEMS array.
 37. The gimbalised microphone of claim35, wherein the microphone is an end-fire line microphone array.
 38. Thegimbalised microphone of claim 35 further comprising one or more sensorsto detect directional data corresponding to the relative directions ofthe gimbalised microphone and a mounting to a unmanned aerial vehicle(UAV); and a processing unit configured to use the directional data tofilter the noise data from the target audio data.
 39. The gimbalisedmicrophone of claim 35 wherein the noise data is filtered from thetarget audio data without a relative direction of the gimbalisedmicrophone and a mounting to a unmanned aerial vehicle (UAV). 40.(canceled)